The light which is incident on the human eye to enable vision is a portion of the radiant energy due to illumination which has been reflected by an object which is seen and has propagated through the air; although the human vision system cannot directly measure the characteristics of objects and illumination, objects can be identified with a degree of reliability even under illumination having unknown chromatic characteristics. This property is called color constancy, and for example enables a white object surface to be perceived as white.
On the other hand, in digital still cameras, digital video cameras and other electronic image-pickup equipment, scenes are picked up as images through the response of a CCD (Charge Coupled Device) or other photosensor; however, because in general the balance of sensor response among the R, G, B, or other color channels is constant, in order to form an image in a state in which the appearance is natural in accordance with the scene illumination, a correction mechanism is necessary to adjust the balance between channels. If the balance is not adequately adjusted, to the viewer of the image, places normally recognized as achromatic objects will be reproduced as colored in the image, or objects will be reproduced with a color different from the color remembered, so an unnatural impression is imparted; hence balance adjustment is extremely important for color reproduction of an image.
Balance adjustment among channels can be performed by, for example, correction of achromatic colors called white balance in the gain adjustment of each channel; by correcting the color rendering properties of the light source through linear matrix transformation of signals among channels (Patent Reference 1); or by matching to the different sensitivity responses of the sensors of image-pickup equipment, vision systems and similar. However, whichever method is used, the correction mechanism must use some means to obtain correction parameters appropriate to the scene. For example, the following equations (1) and (2) can be used to calculate appropriate gain values for adjustment of the white balance of sensors with a RGB three-channel response which is linear with respect to the quantity of light, together with the spectral sensitivity characteristics of the image-pickup system, if the spectral distribution of the light source for the photographed scene is known.
                              [                                                                      R                  w                                                                                                      G                  w                                                                                                      B                  w                                                              ]                =        SI                            (        1        )            
where S is a matrix indicating the sensor sensitivity (three channels×number n of wavelength samples), and I is a column vector indicating the spectral distribution of the light source (number n of wavelength samples).gR=Gw/Rw,gG=Gw/Gw=1.0,gB=Gw/Bw   (2)
However, for the image-pickup equipment, information relating to objects existing in the scene at the time of image pickup without calibration or the like and the illuminating light sources of the scene are normally unknown; and adjustment parameters appropriate to the scene, or the chromatic characteristics of the illuminating light source necessary to determine those parameters, must be identified from the response results of a dedicated sensor or sensor for image pickup, constituting a problem known as the light source estimation problem or the color constancy problem.
In the field of vision studies, various algorithms and calculation models began to be proposed from around 1980, and apart from these, techniques based on empirical knowledge have been incorporated in conventional color image-pickup equipment, the estimation performance of which has advanced through the years. Recently, applications to robotics and other artificial vision systems have also been anticipated.
One of the most widely used algorithms extracts the color components of the light source from average values of sensor response and the projection thereof onto a black body locus, based on the assumption that the spatial average over the scene of the surface reflectivity of an object is close to gray (Non-patent Reference 1, Non-patent Reference 2), and is used in a variety of modes, such as simply averaging the sensor response among pixels, averaging pixels within the range of a specified brightness level, or changing the range or weighting of sampling depending on the position in space. There are also a method in which color components of the light source are extracted from sampling results for pixels with high response values, assuming that the area with the highest brightness level corresponds to a white surface close to a perfectly diffuse reflecting surface (Patent Reference 2), and a method in which an area of high brightness level is assumed to be a specular reflecting component, and the light source is estimated from the distribution of the response values (Non-patent Reference 3). Because these methods are based on an assumption about an object surface, which should be physically independent of the light source, it is known that depending on the scene, the results of light source estimation may be greatly affected by the state of an object which deviates from the assumptions made.
There are also a study in which, by assuming a reflection model in which an object surface is a diffuse reflecting surface, and approximating the spectral characteristics of the light source and of the object surface by a linear model of few dimensions, reconstruction is attempted through linear calculations using a vector space different from that of the sensor response (Non-patent Reference 4), and a study in which constraining conditions, such as that the spectral reflectivity of an object surface must physically be in the range 0 to 1, are applied to select a light source with high probability (Non-patent Reference 5); however, in generalized image-pickup systems with few response channels, these do not independently provide sufficient estimation performance. Further, although the volume of computations is increased, there has also been proposed a method of integrating a plurality of known assumptions and probabilistic distributions for the light source, object surfaces, image-pickup system and similar, to improve the accuracy of statistical estimation (Non-patent Reference 6).
In methods which apply reflection models in particular, rather than performing an estimate taking as the solution a single completely unknown light source, in some methods wide prior knowledge is utilized in a method of determination in which the most probable light sources are categorized or detected from among a number of light sources selected in advance as candidates; such methods may be advantageous in that calculations are comparatively simple and results can be output rapidly. As criteria for judging the reliability of the result, errors by restoring the sensor response itself under a fixed constraint condition may be used (Non-patent Reference 7) ; and there have been proposals for widely using distribution states in the color gamut within the sensor space, to efficiently quantify a correlation relationship through comparison with a color gamut, adopted in advance as a reference, or a weighted distribution (Non-patent Reference 8, Non-patent Reference 9, Non-patent Reference 10, Patent Reference 3).
Patent Reference 1: Published Japanese Patent Application No. 2002-142231
Patent Reference 2: Published Japanese Patent Application No. H9-55948
Patent Reference 3: Published Japanese Patent Application No. H5-191826
Non-patent Reference 1: G. Buchsbaum, “A Spatial Processor Model for Object Color Perception”, J. Franklin Inst., 310, 1980
Non-patent Reference 2: E. H. Land, “Recent Advances in Retinex Theory”, Vision Research, 26, 1986
Non-patent Reference 3: H. C. Lee, “Method for computing the scene-illuminant chromaticity from specular highlights”, J. Opt. Soc. Am. A, Vol. 3, No. 10, 1986
Non-patent Reference 4: L. T. Maloney & B. A. Wandell, “Color Constancy: A method for recovering surface spectral reflectance”, J. Opt. Soc. Am. A, 1986
Non-patent Reference 5: D. A. Forsyth, “A Novel Algorithm for Color Constancy”, Int. J. Comput. Vision, 5, 1990
Non-patent Reference 6: D. H. Brainard & W. T. Freeman, “Bayesian color constancy”, J. Opt. Soc. Am. A, Vol. 14, No. 7, 1997
Non-patent Reference 7: B. Tao, I. Tastl & N. Katoh, “Illumination Detection in Linear Space”, Proc. 8th Color Imaging Conf., 2000
Non-patent Reference 8: Hewlett-Packard Company, Hubel et al., “White point determination using correlation matrix memory”, U.S. Pat. No. 6,038,339
Non-patent Reference 9: G. D. Finlayson, P. M. Hubel & S. Hordley, “Color by correlation”, Proc. 5th Color Imaging Conf., 1997
Non-patent Reference 10: S. Tominaga & B. A. Wandell, “Natural scene-illuminant estimation using the sensor correlation”, Proc. IEEE, Vol. 90, No. 1, 2002